In digital cameras a lens is used to obtain light information from a scene. This light information is converted to electrons by an image sensor having an array of light sensitive elements. When using digital RGB still cameras, three image sensors are used to obtain color sensing for each color red, green and blue. In order to reduce both cost and size of the camera it is possible to use one image sensor having an RGB Bayer filter array, where each pixel in the sensor array senses red, green or blue in a predefined pattern. This pattern is build up of alternating green, red columns and green, blue columns. When using one image sensor to sense all three colors it is necessary to reconstruct the missing pixels before the total image is represented.
When using RGB Bayer sensors so-called green non-uniformity or green-green differences are introduced, because of the differently colored neighbors of a green pixel. In the vertical direction of a ‘red’ column a green pixel has only red colors as a neighbor, while in ‘blue’ column green has only blue neighbor pixels. Depending on the quality of the vertical color separation of the sensor, the green pixels can be modulated by the red and blue pixels due to the electron charge in the silicon layer or light crosstalk in the color filter layer. If so, then for certain colored scenes this will result in a column wise green non-uniformity.
In FIG. 1 an example of the green-green differences is shown according to the above description and it is shown how the column wise modulation of green by the red and blue pixels appears. In FIG. 1, this difference is illustrated as a normal G, and an italic and bold G. In the image signal this can be compensated for when reconstructing the green pixel using some special reconstruction methods.
A method of restoring green uniformity in a reconstructed green signal is described in the following. In case of an RGB reconstruction with two row delays, FIG. 2 shows the declaration of the surrounding green pixels if green is present (left) or absent (right). In the latter case the missing green pixel Gc is the reconstructed (smart) green value of the RGB reconstruction filter. The amount of green non-uniformity depends on the design of the image sensor. For the Philips FT sensors the maximum is about 5% of the green signal amplitude. This level will be mentioned in the ‘Gnonuniformlevel’ and should be adjustable by means of the core of the camera processing.
An algorithm for green uniformity restoration is:
if presence of Green thenbeginsigmaG = (G1+G2+G3+G4)/4if abs(sigmaG−Gc) < Gnonuniformlevel {subtract sum of pixelsand center pixel}then Green = (Gc+sigmaG)/2 {the average green value of 5 pixels}endelse if absence of green thenbeginsigmaG = (G1+G2+G3+G4)/4if abs((G1+G2−G3−G4)/2) < Gnonuniformlevel {subtract horizontaland vertical pixels}then Green = sigmaG {the average green value of 4 pixels}else Green = Gc (the reconstructed green)end
The result of this algorithm is that the green uniformity is restored in the reconstructed signal in such a way that no regular pattern due to the green non-uniformity is visible and that for signal variations above the Gnonuniformlevel no resolution loss will occur. Below that level the center green pixel is replaced by the averaged green value of all present green pixels.
Contour filtering is another important part of image processing, by contour filtering a contour signal is obtained from the contour information of the image. This signal can afterwards be applied to the reconstructed image signal whereby the contours of the image are amplified. In practical use a contour amplification of an image results in a sharper image.
Digital still color cameras with an RGB Bayer color filter array can be equipped with a two-dimensional (2D) contour filter which is acting parallel to the RGB color reconstruction. The advantage of this method is that contour processing after the reconstruction of the RGB signals has become superfluous. Parallel contour processing is especially of interest for low cost applications. No extra row delays are needed for the two-dimensional contour signal because with the very same row delays as applied for the RGB color reconstruction, the contour signal is achieved from the green color only.
Therefore in order to get the best result the green uniformity should not only be restored in the reconstructed data as described above, but green non-uniformity should also be removed from the parallel contour signal.
First the basics of parallel contour processing will be explained. The most important aspect of parallel contour filtering for RGB Bayer image sensors is the so-called zero switch box. The serial RG signal, or GB signal in the next row of the sensor, is de-multiplexed into three RGB signals with zeros on the missing color location.
So the red signal becomes:
R0R0R00000R0R0R00000The blue signal becomes:
000000B0B0000000B0B0And the green signal:
0G0G0G0G0G0G0G0G0G0GEach of those signals can now easily be filtered with ordinary Laplacian filters. In FIG. 3 an example is given how the filter coefficients relate to the presence or absence of center green pixels for an arbitrary 5×3 contour signal. At the bottom of FIG. 3, the coefficients of the total 5×3 filter is shown. At the top left the 5×3 zero switched green pattern with center green absent is shown and below the corresponding filter coefficients. At the top right the 5×3 zero switched green pattern with center green present is shown and below the corresponding filter coefficients.
It is important that the combination of both filters leads to a minimum of distortion in the contour signal. In many cases this can be achieved by determining the coefficients in such a way that the sum of the positive (or negative) coefficients for center green absent, and for center green present, are equal.